Theoretical Statistics and Asymptotics

Cox and Hinkley’s Theoretical Statistics (1974) was arguably the first modern treatment of the foundations and theory of statistics, and for the past thirty years has served as a unique book for the study of what used to be called mathematical statistics. One of its strengths is the large number of examples that serve not only to illustrate key concepts, but also to illustrate limitations of many of the classical approaches to inference. Since the time of its publication there has been considerable development of the theory of inference, especially in the area of higher order likelihood asymptotics. The paper by Barndorff-Nielsen and Cox (1979) read to the Royal Statistical Society initiated many of these developments. There have also been very dramatic advances in the implementation of Bayesian methodology, particularly with the development of sampling methods for computing posterior distributions. A current approach to the study of theoretical statistics should be informed by these advances. Barndorff-Nielsen and Cox (1994) presents an approach to inference based on likelihood asymptotics, as does Pace and Salvan (1997). Severini (2000a) covers much of the same material at a somewhat more elementary level. Bayesian methodology is covered in books on that topic, for example Bernardo and Smith (2000), while Schervish (1995) presents an encyclopedic treatment of both classical and Bayesian approaches to inference. Davison (2003) provides a masterful and wide-ranging account of the use of statistical models in theory and practice, with a welcome emphasis on many practical aspects, and this approach is closest in spirit to the one I would like to follow. At a more advanced level there is not to my knowledge a treatment of the main themes of theoretical statistics that attempts to incorporate recent advances in a unified framework. In this paper I will attempt to set out some ideas for such a framework, building on topics and examples from Cox and Hinkley (1974), and very heavily influenced by Barndorff-Nielsen and Cox (1994). The emphasis is on some theoretical principles that have their basis in asymptotics based on the likelihood function. There are of course many important advances and developments in statistical theory and methodology that are not subsumed by asymptotic theory or Bayesian inference. These include developments in nonparametric and semiparametric inference as well as a large number of more specialised techniques for particular applications. The explosion in statistical methods in a great many areas of science have made a comprehensive study of the field of statistics much more difficult than perhaps it was, or seems to have been, thirty years ago. In light of David Cox’s view that a theory of statistics forms a basis both for understanding new problems and for developing new methods, I try here to focus on how to structure this theory for students of statistics. In §2 I sketch a framework for the implementation of higher order asymptotics and Bayesian inference into my ideal course or text on the theory of statistics. In §3 I note topics from a more standard treatment of theoretical statistics that have been omitted and provide some brief conclusions.